The use of large data centers to provide a computing infrastructure has become widespread. These data centers may experience “peak load” periods where the demand on the infrastructure drastically increases for short periods of time.
The average data center infrastructure is designed to handle an average computational load, but it is under provisioned for peak loads. When a peak occurs, workloads take more time to complete. Mildly over provisioning the infrastructure of a data center above its average computational load does not solve the problem of noticeably slower application performance during peak loads. For example, if a data center with a 10% over provision for average computational load has a peak load that requires 50% additional capacity, a task that should finish in two weeks would take ten weeks to complete. In many cases, such delays cannot be tolerated by a business because tasks occurring during peak loads are typically time sensitive (e.g., end of quarter calculations, holiday on-line shopping, or a tape-out activity associated with releasing a new chip). As a result, other mainline activities are pushed out or delayed, leading to overall business disruption.
Additional computing resources are also typically needed to account for hardware failure and unforeseen new workloads. To minimize a service disruption after server failure, a data center may use an additional standby server to take over the workloads of the faulty server and continue to serve client requests. Similar high availability systems are needed when an administrator of a data center has unforeseen new workloads, even if they are not characterized as peak loads. Examples of unforeseen new workloads include new projects and testing new tools for which extra computing capacity is needed.
The main approach to handling these difficulties is to over provision a data center's infrastructure by maintaining substantially more hardware than is needed. This over provisioning of the infrastructure of the data center causes performance to never noticeably be affected by peak loads because the hardware is rarely fully utilized. For example, if a data center with a 50% over provision for an average computational load has a peak load that requires the additional capacity, but only for 20% of the time, then 40% of the infrastructure is wasted on average, since 80% of the time, 50% of capacity is not utilized. Over provisioning in this manner decreases profit margins significantly by increasing base costs of the infrastructure, and recurring costs of power, cooling, maintenance, hardware refresh, software licenses and human resources.
The prior examples also assume peak loads and average computational load are semi-predictable when in fact they are not. Peak loads can be longer in time, higher in computing needs, and at time completely unexpected. For instance, a “trading storm” due to an unforeseen event that creates turmoil in the stock market leads to unpredictable peak load. The difference between average loads and peak loads can become convoluted as a data center experiences new projects, changes in business, growth or decline in headcount, and additional capacity requirements of existing projects. The inability to forecast computational needs of a data center regularly leads to higher costs, slower performance, excessive downtime, or even underutilization of available resources.
The approaches described in this section are approaches that could be pursued, but are not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.